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Abstract —Support recovery of sparse signals from compressed 
linear measurements is a fundamental problem in compressed 
sensing (CS). In this paper, we study the orthogonal matching 
pursuit (OMP) algorithm for the recovery of support under noise. 
We consider two signal-to-noise ratio (SNR) settings: i) the SNR 
depends on the sparsity level K of input signals, and ii) the SNR 
is an absolute constant independent of K. For the first setting, we 
establish necessary and sufficient conditions for the exact support 
recovery with OMP, expressed as lower bounds on the SNR. Our 
results indicate that in order to ensure the exact support recovery 
of all N-sparse signals with the OMP algorithm, the SNR must 
at least scale linearly with the sparsity level K. In the second 
setting, since the necessary condition on the SNR is not fulfilled, 
the exact support recovery with OMP is impossible. However, 
our analysis shows that recovery with an arbitrarily small but 
constant fraction of errors is possible with the OMP algorithm. 
This result may be useful for some practical applications where 
obtaining some large fraction of support positions is adequate. 

Index Terms —Compressed sensing (CS), orthogonal matching 
pursuit (OMP), restricted isometry property (RIP), signal-to- 
noise ratio (SNR), minimum-to-average ratio (MAR). 

I. Introduction 


quite simple: it iteratively identifies the support of the sparse 
signal, by adding one index into the list at a time according to 
the maximum correlation between columns of measurement 
matrix and the current residual. There are several popular 
stopping rules for the OMP algorithm that can be implemented 
at minimal cost [10]: 

i) Stop after a fixed number of iterations: k = K. 

ii) Stop when the energy in the residual is small: l|r fc || 2 < e. 

iii) Stop when no column in the measurement matrix is 
strongly correlated with the residual: | <f>' r fc_1 ||oc < £■ 

See Table I for the mathematical description of a version of 
OMP. Both in theory and in practice, the OMP algorithm has 
demonstrated competitive performance [11]. 

Over the years, many efforts have been made to analyze the 
performance of OMP in sparse support recovery. In one line 
of work, probabilistic analyses have been proposed. Tropp and 
Gilbert showed that when the measurement matrix $ is gen¬ 
erated iid at random, OMP can ensure the accurate recovery 
of every fixed /C-sparse signal from noise-free measurements 
with overwhelming probability with [11] 


We consider the support recovery of a high-dimensional 
sparse signal from a small number of linear measurements. 
This is a fundamental problem in compressed sensing (CS) [1], 
[2] and has also received much attention in the fields of 
sparse approximation [3], signal denoising [4], and statistical 
model selection [5]. Let x £ lZ n be a AT-sparse signal (i.e., 
||x|| 0 < K -C n) and $ £ 77. mxrl n ) b e ^e 

measurement matrix. The measurements are given by 

y = <Ex + v (1) 

where v is the corrupting noise. The goal of support recovery 
is to identify the support (i.e., the positions of nonzero ele¬ 
ments) of the input signal x from the measurement vector y. It 
is known that optimal support recovery requires an exhaustive 
search over all possible support sets of the sparse signal and 
hence is NP-hard [6]. For this reason, much attention has 
been drawn to computationally efficient approaches. In this 
paper we consider the orthogonal matching pursuit (OMP) 
algorithm for solving the support recovery problem. OMP is a 
canonical greedy algorithm for sparse approximation in signal 
processing [7], [8]. It is also known as greedy least square 
regression in statistics [5] and forward greedy selection in 
machine learning [9]. The principle of the OMP algorithm is 


m > cK log n 


( 2 ) 


for some constant c. Fletcher and Rangan provided an im¬ 
proved scaling law on the number of measurements and also 
showed that the scaling law works for noisy scenarios for 
which the signal-to-noise (SNR) goes to infinity [12]. 

Another direction is to develop deterministic conditions for 
the exact support recovery with the OMP algorithm [13] — [21]. 
Those conditions are often characterized by the properties of 
measurement matrices, such as the mutual incoherence prop¬ 
erty (MIP) [22] and the restricted isometry property (RIP) [23]. 
A measurement matrix $ is said to satisfy the RIP of order 
K if there exists a constant c £ [0,1) such that 

(1 - c)||x||l < H^xHl < (1 + c)||x||l (3) 


for all A-sparse vectors x. In particular, the minimum value 
among all constants c satisfying (3) is called the isometry 
constant 5k- In the noise-free case (i.e., when the noise vector 
v = 0), Davenport and Wakin showed that [14] 


5k +i < 


1 

3 VK 


(4) 


is sufficient for OMP to accurately recover the support of 
the input signal. For further improvements on this condition. 
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TABLE I 

The OMP Algorithm 


Input <I>. y, and sparsity level K. 

Initialize iteration counter k = 0, 

estimated support T° = 0, 
and residual vector r° = y. 

While k < K do 
k = k + 1. 

Identify 1 t k = argmax | , r fc—1 ) |. 

ten\T '=- 1 

Enlarge T k =T k ~ 1 Ut k . 

Estimate x fc = argmin ||y — <f>u|| 2 . 

u:supp(u)='7~k 
Update r k = y — ^xL 

End 

Output the estimated support T K and vector x A . 


see [15]—[20], [24], [25]. The deterministic conditions on the 
exact support recovery with OMP in the noisy case have been 
studied in [9], [25]—[28], in which the researchers considered 
the OMP algorithm with residual-based stopping rules and 
established sufficient conditions for the exact support recovery 
that depend on the properties of measurement matrices and the 
minimum magnitude of the nonzero elements of the signal. 

The main purpose of this paper is to investigate deter¬ 
ministic conditions of OMP for the support recovery in the 
noisy case. Unlike previous studies that considered residual- 
based stopping rules for the OMP algorithm, we simply 
consider that OMP runs K iterations before stopping, which 
is arguably the most natural stopping rule if one is concerned 
with the recovery of exact support. We establish necessary and 
sufficient conditions for the exact support recovery with OMP, 
expressed as lower bounds on the SNR. Our results indicate 
that the OMP algorithm can accurately recover the support 
of all ff-sparse signals only when the SNR scales at least 
linearly with the sparsity K. For high-dimensional setting, this 
essentially requires the SNR to be unbounded from above. 

We also study the situation where the SNR is upper bounded 
so that the necessary condition for the exact support recovery 
with OMP is not fulfilled. The analysis of OMP with bounded 
SNR has been an interesting open problem [12]. We consider 
a practical case where the SNR is an absolute constant 
independent of the sparsity K. Our result shows that under 
appropriate conditions on the SNR and the isometry constant, 
OMP can approximately recover the support of sparse signals 
with only a small constant fraction of errors. 

The main contributions of this paper are summarized as 
follows. 

i) We consider the exact support recovery with OMP in 
the noisy scenario. Our analysis shows that OMP can 
accurately recover the support of any /t-sparse signal if 


v / SNR > 


2VK(1 + S k +i) 
(i-(V^ + i)^ + i)-Vmar’ 


(5) 


1 There is another popular version of OMP in the literature which normalizes 
columns of & before the identification step (see, e.g., [12]). In the present 
paper we exclusively consider the version of OMP without column normal¬ 
ization, since results obtained for this version can be readily extended to the 
version with normalization via rescaling. Note that when has normalized 
columns (e.g., dictionary atoms), the two versions of OMP become identical. 


where MAR is the minimum-to-average ratio of the input 
signal (see definitions of the MAR and the SNR in 
Section III). We also establish a necessary condition for 
the exact support recovery with OMP as 


\/SNR > 


'/K(l + Sk+ i ) 

(1 - VK5 k + i) • \/MAR 


( 6 ) 


Therefore, to ensure the perfect support recovery of all 
/v-sparse signals with OMP, the SNR must at least scale 
linearly with the sparsity level K of input signals. 


ii) We also consider the case where SNR is an absolute con¬ 
stant independent of the signal sparsity K. Our analysis 
shows that if 

VsM>-0j (7) 

S 2K 

where k := max, i j es „ ;)p ( x ) j^j, then OMP can recover 
the support of any A'-sparse signal with error rate 

Perror < C/t 2 ^, ( 8 ) 

where C is a constant. Therefore, with properly chosen 
isometry constants, the fraction of errors in the support 
recovery can be made arbitrarily small. 


The rest of this paper is organized as follows: In Section II, 
we introduce notations and lemmas that are used in this paper. 
In Section III, we analyze necessary and sufficient conditions 
for the exact support recovery with OMP. In Section IV, 
we provide the results of OMP for the approximate support 
recovery of sparse signals. Concluding remarks are given in 
Section V. 


II. Preliminaries 

A. Notations 

We briefly summarize notations used in this paper. Let V. = 
{ 1 , 2 ,-•• ,n} and T = suppipL) = {i\i S f l,Xi ^ 0 } denote 
the support of vector x. For S C ft, |<S| is the cardinality of 
S. T \ S is the set of all elements contained in T but not 
in S. xs € AJ 6 ' represents a restriction of the vector x to 
the elements with indices in S. G 72 rnx l‘ s l is a submatrix 
of $ that only contains columns indexed by S. If $5 is full 
column rank, then is the pseudoinverse 

of $ 5 . span(<F,s) represents the span of columns in $ 5 . P 5 = 
stands for the projection onto span(<l? 5 ). P^r = I — P 5 
is the projection onto the orthogonal complement of spanks), 
where I denotes the identity matrix. 

B. Lemmas 

The following lemmas are useful for our analysis. 

Lemma 2.1 (Lemma 3 in [23]): If a measurement matrix 
satisfies the RIP of both orders K\ and K 2 where J\i < K 2 , 
then 

Sk! < Sk 2 - 

This property is often referred to as the monotonicity of the 
isometry constant. 
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Lemma 2.2 (Direct consequences of RIP [29], [30]): Let 
S C fl. If (S| 5 | < 1 then for any u £ , 

(1 - <5| S |) ||u|| 2 < ||$ 5 $ s u|| 2 < (1 + $|5|) ||u|| 2 , 

—||u|| 2 < ||($^ s )- 1 u|| 2 < -J— ||u|| 2 . 

1 + 0|5| 1 - d|5| 

Lemma 2.3 (Consequences of RIP [31], [32]): Let 

5 i,<S2 ,S3 — ^ anc * n <S>2 n S 2 = 0- if £|Si|+|s 2 |+|s 3 | < i> 

then for any vector v £ TZ) S ^, 

II^5i^S 2 v I|2 < %i| + |5 2 | IMI 2 > 
II^5 i P 5 3 ^5 2 v|| 2 < <5|Si|+|S 2 | + |S 3 | IMI 2 • 

Lemma 2.4 (Proposition 3.1 in [29]): Let S C Cl. If <5|<s| < 
1, then for any vector u £ lZ m , 

Il^5 u ll2 < + <5 |siII u I|2- 

Lemma 2.5 (Lemma 5 in [26]): Let Si, 1S2, C Q be disjoint 
sets and denote A = and B = uS2 $ S| u s 2 . 

Then the minimum and maximum eigenvalues of A and B 
satisfy 

Amin(A) ^ A m in(B) and A m ax(A) ^ Amax(P)- 

III. Exact Support Recovery via OMP in the 
Presence of Noise 

A. Main Results 


where 

2 s/K 

v/SNR • MAlf 

Hence, when t —> 0, the condition reduces to 

Sk+1 < —7’ C 11 ) 

which coincides with the recovery condition of OMP in the 
noise-free case [19], [20]. The condition in (11) has also 
been shown to be nearly necessary for the exact support 
recovery with OMP since it cannot be further relaxed to 
Sk+i < jz [19], [20], [33], 

The following theorem gives a necessary condition for the 
exact support recovery with the OMP algorithm. 


Theorem 3.2 (Necessary Condition): If one wish to accu¬ 
rately recover the support of any A'-sparse signal x from its 
noisy measurements y = d?x + v with OMP, then the SNR 
should satisfy 


v / SNR > - V ^ (1 + ^ +1 / ) . 

(1 - Vks k + 1 ) • Vmar 

Proof: See Appendix A. 


( 12 ) 


Remark 1: 

i) Loosely speaking, the lower bounds in (9) and (12) can 
be matched within a constant factor of two. Interestingly, 
since MAR < 1 and Sk+i > 0, one can directly obtain 
from (12) that 

SNR > K, (13) 


In this section, we analyze the condition for the exact sup¬ 
port recovery with OMP in the presence of noise. Following 
the analysis in [12], we parameterize the dependence on the 
noise v and the signal x with two quantities: 


SNR 



and 


MAR = 


minjg r \xj\ 2 

11*11 l/K 


The next theorem provides a condition under which OMP can 
accurately recover the support of any A'-sparse signal x. 


Theorem 3.1 (Sufficient Condition): Suppose that the mea¬ 
surement matrix $ satisfies the RIP with 8 k +1 < -7=-^. 
Then OMP performs the exact support recovery of any A'- 
sparse signal x from its noisy measurements y = <f>x + v, 
provided that 


v/snr >_+ _ 

( 1 - (Vk + i)5 k +i)- Vmar 


(9) 


One can interpret from Theorem 3.1 that in the high¬ 
dimensional setting, the exact support recovery with OMP can 
be ensured in the high SNR region. Moreover, observe that (9) 
can be rewritten as 


Sk+ 1 < 


1 - t 

VK+l + t’ 


( 10 ) 


which implies that in order for OMP to accurately recover 
the support of any A'-sparse signal, the SNR must at 
least scale linearly with the sparsity level AT of the signal. 
For high-dimensional setting, this essentially requires the 
SNR to be unbounded from above. 

ii) The sufficient condition for the exact support recovery 
with OMP under a similar SNR setting has been studied 
in [12], in which the authors considered large random 
measurement matrices. They provided asymptotic and 
probabilistic results on the scaling law for the number of 
measurements that ensures the exact support recovery of 
every fixed AT-sparse signal when the SNR approaches to 
infinity. Our result in Theorem 3.1 extends that in [12] by 
providing a deterministic condition that applies to general 
measurement matrices and also holds uniformly for all 
AT-sparse signals. 

iii) The result in Theorem 3.1 is also closely related to the 
results in [9], [25]-[28], in which the authors considered 
the OMP algorithm with residual-based stopping rules by 
assuming that the noise level is known a priori. They 
established conditions that depend on the properties of 
measurement matrices and the minimum magnitude of 
nonzero elements in the signal x. In fact, it can be shown 
that the conditions proposed in [9], [25]-[28] essentially 
impose a similar requirement on the size of measurements 
as our result established in Theorem 3.1. However, our 
analysis differs in 1) that we characterize the sufficient 
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condition for the exact support recovery with a lower 
bound on the SNR and 2) that we consider the stopping 
rule that the OMP algorithm runs exact K iterations 
before stopping and does not require the assumption 
of knowing the noise level. In addition, we provide in 
Theorem 3.2 the necessary condition analysis for the 
exact support recovery with OMP, for which there is no 
counterpart in the studies of [9], [25]—[28] . 

iv) For the OMP algorithm, it had become usual, when 
recovering a K -sparse signal, to consider the performance 
of the algorithm after K iterations. See, for instance, [11]- 
[15], [19], [20], [25]. Compared to stopping rules that are 
based on the residual tolerance (e.g., [26]) or the number 
of maximally allowed iterations (e.g., \cK~\ iterations, 
c > 1 [17], [18], [34]—[36]), running OMP for K 
iterations is very natural in that it directly allows the 
algorithm to recover the exact support set without false 
alarms or missed detections since the K -th iterate is itself 
a A'-sparse signal. This feature is appealing when one is 
concerned with the exact support recovery. 

v) It is worth mentioning that for the noise-free case, the 
stopping rule of OMP does not require additional infor¬ 
mation about the data, since it can simply be ||r fc ||2 = 0 
(or ||$ , r fc_1 || 00 = 0). 2 Whereas, in the noisy case, some 
prior information on the signal and the noise is often 
needed. For example, residual-based stopping rules may 
rely on the prior knowledge of the noise level [9], [25]— 
[28], and running OMP for K iterations requires to know 
a priori the sparsity level K of the input signal. The 
assumption of knowing the sparsity (or its region) has 
been commonly made for the algorithm design and the 
performance analysis in the CS field (see, e.g., [29], 
[37] — [41]). However, it should be noted that in many 
applications, the sparsity K is often not available and 
moreover, the underlying signal may not be exactly 
sparse. A typical scenario is that the input signal is not 
exactly sparse but only approximately sparse with a few 
significant nonzero coefficients. 3 For this scenario, our 
analysis for the recovery of the exactly sparse signal can 
be readily extended to the recovery of significant nonzero 
coefficients of the signal, by treating the contribution 
of those small nonzero coefficients as part of noise and 
utilizing the techniques developed in [26], [41]. 


B. Proof of Theorem 3.1 

Our proof of Theorem 3.1 is essentially an extension of 
the proof technique in [42, Theorem 3.4 and 3.5]. Note that 
[42, Theorem 3.4 and 3.5] studied the recovery condition 
for the generalized OMP (gOMP) algorithm in the noise- 
free situation. Our contribution is to generalize the analysis 
to the noisy case. We would like to mention that [42] also 

2 Without noise, the stopping rule of ||r fc H 2 = 0 (or ||<i? , r fe-1 || 00 = 0) is 
essentially equivalent to running OMP for K iterations. 

3 Note that if the signal to be recovered is not even approximately sparse, 
then compressed sensing technique may not apply. 


provided a noisy case analysis but focused only on the t 2 - 
norm distortion of the signal recovery and the corresponding 
result is also weaker than the result established in this paper 
(see Section III-C). 

The proof works by mathematical induction. For the conve¬ 
nience of stating the results, we say that OMP makes a success 
at an iteration if it selects a correct index at the iteration. 
We will first give a condition that guarantees the success of 
OMP at the first iteration. Then we will assume that OMP 
has been successful in the previous k (1 < k < K) iterations 
and will derive a condition under which OMP also makes a 
success at the (fc+l)-th iteration. Finally, we will combine the 
two conditions to establish an overall condition for the OMP 
algorithm. 

« Success at the first iteration: 

From Table I, we know that at the first iteration, OMP 
selects the index t 1 corresponding to the column o t i £ $ 
that is most strongly correlated with the measurement 
vector y. Hence, 


|(^i,y)| = max|(^,y)| 
> max |(0j,y}| 

- 


(a) 

1 

s/K 

|| <1? ^-$7^X7- + 

•v || 2 

W 

> 

1 

Vk 

(1 ^r^r x r b - 

ll*rv||2) 

(c) 

1 



> 

7k 

((l-<S*)||x|| 2 - 

ll*rv||2) 

(d) 

> 

1 

s/K 

((l-fo)||x|| 2 - 

\A + 8k 11 V 11 2 J 


(14) 


where (a) is because y = <hx + v, (b) is from the 
triangle inequality, (c) is from the RIP, and (d) is due 
to Lemma 2.4. 

On the other hand, if a wrong index is chosen at the first 
iteration (i.e., t 1 £ T), then 


y)l 


= |<&i,$x) + (&l,v)| 

< |(0fi,$x)| + |(0fi,v)| 

= + IKiv|| 2 

( b) 

< s K +1 i|x|| 2 + n<y tl v|| 2 

< Sk+ 1 ||x|| 2 + y/l + <5l||v|| 2 , (15) 


where (a) uses the triangle inequality, (b) follows from 
Lemma 2.3 and (c) is due to Lemma 2.4. 
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This, however, contradicts (14) whenever 

^ if + l|| x ||2 + \A + $1 || v ||2 

< ((1 - MIMk - \/l + 6 k ||v|| 2 ) • (16) 

Since S± < 5k < 6k+ i (by the monotonicity of isometry 
constant), (16) is guaranteed by 


<5fS'+l|| x ||2 + \A + +111 v 112 

< ((! - ^ ir + i ) ll x ll2 - yrrwiMb ) ■ ( 1 7 ) 

Or equivalently, 

(t-H " 51 " 2 

> (l + -L) Vl + fo+iMa. (18) 

Furthermore, since 

||$x|| 2 < \A + 5 k ||x|| 2 < 1/ 1 + 8 k +\ ||x|| 2 , (19) 

and also noting that SNR = ^*| 2 , we can show that 
(18) holds true if 


1 — Sk+i 
s/K 
That is. 


$ k +1 > 


1 + Sk+ 1 

VSNR 



VSNR > (v^+^ + ^+O 

i-(Vk + i) 6 k+1 


( 20 ) 


( 21 ) 


Therefore, under (21), a correct index is chosen at the 
first iteration of OMR 


# Success at the general iteration: 

Assume that OMP has been successful in each of the 
previous k (1 < k < K) iterations. Then, 

|T nT fe | = k. (22) 

Under this assumption, we will derive a condition that 
ensures OMP to make a success at the (k + l)-th iteration 
as well. 

For analytical convenience, we introduce two quantities. 
Let u denote the largest value in {\{4>i, ^ k )\}ier\T k ar *d 
let v denote the largest value in {\(4>i, r fc }|}iefi\(TuT fc )- 
Note that T\T k and fl\(TUT fc ) are the set of remaining 
correct indices and the set of remaining incorrect indices, 
respectively. Then it is clear that if 

u > v, (23) 

a good index will be selected at the (fc + l)-th iteration of 
OMP. The following proposition characterizes the lower 
bound of u and the upper bound of v. 

Proposition 3.3: 

U ~ y/K - k ~ ^)H X T\T fc ll2 - \/l + , 

V < &K +1 || x 7 - \7 _fe II 2 + + II v|| 2 . 

Proof: See Appendix B. ■ 


Using Proposition 3.3, we obtain the sufficient condition 

to u > v as 

_ k ((! - ^ J ft:)|l x r\T fc II 2 - \A + S K ||v|| 2 ) 

> <)ir+l|| x T\T' ! ll2 + "\/1 + IIv|| 2 , 

or equivalently, 

“ Sk+1 ) l|xnrfc|12 

> (j- + J V 1 + +111 v 112 - (24) 

Note that 


11 x T\T fc 11 


> 


(a) 


\J\T\T k | min |x 


> VK -k 


( Vmar- 


x 2 


\ VK 


( b) 
> 


(c) 


K-k 


K( 1 + Sk+ 1) 

I K-k 

K{ 1 + 6k+ t) 


• VMAR • ||$x|| 


• Vmar-snr-|| v|| 2 , (25) 


where (a) is from the definition of MAR, (b) is from (19), 

and (c) is because SNR = . 

7 IMI l 

Using (24) and (25), we can show u > v holds true if 




K-k 


> 


K( 1 + 8k+i) 

( 1 + VF^) ' /1 + Sk+1 - 


■ Vmar • snr 
(26) 


That is, 
VSNR > 


(l + 6 K+ i)(VK=h+l)VK 


(i - (VK -k + i)S k +i)Vk - k ■ Vmar' 

(27) 

Furthermore, observe that 

(l+8 K+ i)(VK=k. + l)VK 


(i - (VTf - k + i)S K +i)Vi< - k • Vmar 

_ VK( 1 + 8k+i) _ 

(i - (VK~^k + i)S K +i) • Vmar 
i 


X 1 + 


(a) 

< 


VK-ky 
2V~K(1 + 8k+ i) 


(i- (Vk + i)8k+i)- Vmar’ 


(28) 


where (a) is from the assumption that 1 < k < K and 
hence 1 < VK — k < V~K- 

Hence, using (27) and (28), we can show that u > v is 
also ensured by 


VSNR > 


2V~K(l + 8k+ i ) 


(i-(Vk + i)s k+1 )- Vmar’ 


(29) 
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Therefore, under (29), OMP makes a success at the (k + 
l)-th iteration. 

So far, we have obtained condition (21) for the success 
of the first iteration and condition (29) for the success of 
the general iteration. We now combing the two conditions to 
obtain an overall condition that ensures the selection of all 
support indices with the OMP algorithm. 

Clearly the overall condition will be determined by the more 
restrictive one between conditions (21) and (29). Thus we 
compare the right-hand-side of (21) and (29). Since 

2(1 + 5 k +i)Vk > (1 + 6 K+1 )(VK+1) 

(1 - (VK + 1)5 k +i) ■ VMAR “ 1 - {VK + l)S K +i ’ 

condition (29) is more restrictive than (21) and hence becomes 
the overall condition for the OMP algorithm. The proof is thus 
complete. 

C. Recovery Distortion in i^-norm 

When all support indices of x have been recovered with 


OMP (i.e., T a = T), we have 

II K 11 ( a ) 

||x-x || 2 = 

ll xt* - & T Kyh 

= 

II X T* - $\-K (3?T x T + v)|| 2 

A 

|| X T* - ^r K ^T Kyi T K + v )ll 

= 

II$V kV H 2 

(c) 

< 

#T k ^V kV I 2 

\J 1 - $\T K \ 

= 

l|Pr*-v|| 2 
x/1 — 5k 

< 

IMI2 


s/1 — 5 k' 


where (a) is because T K = T and 


x A = argmin ||y—<&u||2, 

u:supp(u)='T K 

(b) is due to T K = T, and (c) is from the RIP. 

One can interpret from (30) that the upper bound of the 1 2 - 
norm recovery distortion with OMP is just proportional with 
the noise energy, which outperforms the result in [42] that sug¬ 
gested a recovery distortion upper bounded by 0 {s/K)\\v\\ 2 . 

IV. Approximate Support Recovery via OMP in the 
Presence of Noise 

In the last section we have shown that the exact support 
recovery with OMP requires the SNR to scale linearly with 
the sparsity lever K. For high-dimensional setting, this would 
require the SNR to be unbounded. However, in practical 
applications, we are often facing with the situation where 
the SNR is bounded from above. A particularly interesting 
case might be the case where the SNR is an absolute constant 
independent of the problem size. In this case, of course, the 
necessary condition (in Section III) is not fulfilled so that 
the exact support recovery of all sparse signals with OMP 


is impossible. However, we will show that recovery with an 
arbitrarily small but constant fraction of errors is possible. 

The following theorem demonstrates that for properly cho¬ 
sen isometry constants, there exists an absolute constant SNR, 
under which OMP can approximately recover the support of 
any A-sparse signal with a small constant fraction of errors. 

Theorem 4.1: Let n := maxjj e 7 - j^j. Then if SNR > 

k 2 S 2 ^ 2 , OMP recovers the support of A-sparse signal x from 
its noisy measurements y = <&x + v with error rate 

n < rvV /2 

PeiTor U 2K 

where C is a constant. 

Remark 2: 

i) It is intuitively easy to see that the bound of error rate 
in Theorem 4.1 is reasonable because in the special case 
of orthonormal matrix 4? (i.e., 4 2 k = 0), the result in 
Theorem 4.1 suggests that if SNR = 00 , then the error 
rate p = 0, which matches with the trivial fact that when 
there is no noise and $ is an orthonormal matrix, OMP 
can identify a correct index at each iteration and will 
accurately recover the whole support of signal x in exact 
K iterations. 

ii) An interesting point we would like to mention is that our 
result for the approximate support recovery with OMP 
only requires the isometry constant 82 K to be an absolute 
constant, which essentially imposes a mild constraint on 
the measurement matrix <F. For example, for random 
Gaussian measurement matrices, it can be satisfied with 

Tl 

m > cK log — (31) 

A 

for some constant c [23], [43]. In CS, this implies that 
OMP can essentially perform the approximate support 
recovery of sparse signals with optimal number of random 
measurements up to a constant. 


It is well known that recovering sparse signals with nonzero 
elements of same magnitude is a particularly challenging case 
for the OMP algorithm [17], [39]. The following corollary 
provides the result of OMP on the approximate support 
recovery for this type of input signals. 

Corollary 4.2: Consider AT-sparse signals x with nonzero 
elements of equal magnitude. Then if SNR > S 2 ^ 2 , OMP 
can recover the support of x from its noisy measurements 
y = 4>x + v with error rate 


Perror < C5 


1/2 
2 K 


where C is a constant. 


Our analysis is inspired by the recent work of Livshitz and 
Temlyakov [36] and will also rely on some proof techniques 
in [35]. The main idea behind our analysis is that most of 
support indices can essentially be identified in K iterations 
of OMP and the number of false alarms is small. The proof 
of Theorem 4.1 follows along a similar line as the proofs 
in [36], but with three important distinctions. Firstly, the main 
goals of proofs are different. While our analysis is based on 
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the approximate support recovery of A-sparse signals with 
K iterations of OMP, the analysis in [36] concerned the 
exact support recovery with OMP in more than K iterations. 
Secondly, compared to the result in [36], our result is more 
general in that it applies to input signals with arbitrary sparsity 
level K and with nonzero elements of arbitrary magnitudes. 
Note that the analysis of [36] assumed that K > S 2K , which 
essentially applies to the situation where the sparsity K of the 
input signal is nontrivial. In addition, [36] considered only the 
recovery of signals with magnitudes of nonzero elements upper 
bounded by one. Thirdly, and most importantly, we consider 
the scenario where the measurement noise is present and build 
conditions based on the SNR. Whereas, the analyses in [36] 
focused only on the situation without noise. 

A. Proof of Theorem 4.1 

Before we proceed to the details of the proof, we introduce 
some useful notations and definitions. For notational simplic¬ 
ity, let 5 := 52K- At the fc-th iteration (0 < k < K), let 
r fe := TXT 1 '' denote the set of missed detection of support 
indices. For given constant r £ (0,1], let T k denote the 
subset of r fc corresponding to the \tK] largest elements (in 
magnitude) of x r r. Also, let x k denote the \tK~\- th largest 
element (in magnitude) in xp*. Following the idea in [36], we 
will fix r = (5 1 / 2 . If \tK~\ > |F fc |, then set Y k = and 
x k T = 0. Since OMP totally runs K iterations before stopping, 
the error rate of the support recovery can be given by 


PeiTOr ■ — 


ir A \n 

|T| 


(32) 


(see Appendix C) 4 

\\r k \\l-\\r k+1 \\l> 

1 + 01 

In the second step, we show that (see Appendix D) 
||*'rX>(l-7r) (x k T ) 2 . 

Using (35) and (36), we have 

1 — 6r 


(35) 


(36) 


fc||2 _ II vA+1 II 2 


r , - r 


> 


2 — l i | x 

1 + Oi 

(“) /1 — 6 T 

> 


1 + <5 

1 — 6r 


= ( 1 - 7r + 

( 6 ) 


(x^) 2 

(Xrf 

T — T 


2 7r 3 


1 + r 2 


k \2 


K) 


> (l-7r)(x^) 2 

= (1-70 1 / 2 ) (x* 1/2 )\ (37) 

where (a) is because 5 = § 2 k > and (b) is from r = (5 1 / 2 £ 
[0,1), which establishes the proposition. ■ 

In Proposition 4.3, we have shown that each iteration of 
OMP makes non-trivial progress by providing the lower bound 
on the reduction of residual energy at each iteration. Next, 
using the bound obtained in Proposition 4.3, we will derive an 
upper bound on the number of missed detections in the support 
recovery of OMP. Without loss of generality we assume that 
T = {1, • • • ,K} and that the elements of {xi} k L 1 are in a 
descending order of their magnitudes. Then from the definition 
of x k we have that for any k > 0, k + \tK~\ < K, 


The proof of Theorem 4.1 consists of two parts. In the 
first part, we will provide a lower bound on the reduction 
of residual energy at each iteration of OMP (Proposition 4.3). 
In the second part, by means of the lower bound obtained 
in Proposition 4.3, we will estimate the remaining energy in 
the residual vector r A . The estimate of the energy of r A will 
then allow us to derive an upper bound on the number of 
missed detections (i.e., | U |). Since the OMP algorithm totally 


|x T | > \ x k+\ tK~\ I- 

By applying Proposition 4.3, we have 


(38) 


r K \\l = llr 


chooses I\ indices, it is easy to see that the number missed 
detections (after K iterations) is equal to the number of false 

(a) 

< 

alarms, i.e., 



|r*j = | r K \n 

(33) 

< 

which implies that 



_ |r*| 

Perror • 

(34) 

(U 

< 


Therefore, from the upper bound of |r K | we can directly 
obtain an upper bound of the error rate for the support recovery 
with OMP. 

Proposition 4.3: For any 0 < k < K — {S 1 / 2 !?], the 
residual of OMP satisfies 


I r fe No — llr fc+1 


Hi > (i - 7<5 1/2 ) (x k sl/2 y 


Proof: We shall prove Proposition 4.3 in two steps. First, 
we show that the residual power difference of OMP satisfies 


°lli - ^(II^Hl - ||r A+1 ||l) 

k =0 

K-\tK 1 

E (ll rfe lli-ll rfc+1 ll 

k =1 

K-\tK 1 


k =1 

K-\tK 1 

< llylli — E (!-7r) {x k+WK y 2 

k =1 

= llylli - E (l-7r)(x i ) 2 (39) 

k= \tK~\ +1 

where (a) uses the facts that \tK] > 1 and that the energy of 
residual of the OMP algorithm is always non-increasing with 
the number of iterations (i.e., ||r fe || 2 > ||r fc+1 || 2 > k > 0), and 
(b) is from (38). 

4 This proof is essentially identical to a result in [35]. Since it will play a 
key role in the proof of (36), we include the proof for completeness. 









Note that 


lly II1 = ||^x + v||I 

< (l + r)||$x||i + (l + l/r)||v||^ 

( 6 ) 

< (1 + t )( 1 - 5 )|| x || 2 + (1 + 1/t)||v|| 2 

< (1 + 3t)||x|| 2 + (1 + 1/r) ||v ||2 

K 

= (1 + 3t) ^2(xi) 2 + (1 + 1/r) ||v|||, (40) 

2=1 

where (a) is from the fact that 

l|u + v||l<(l + r)||u||| + (l + l/r)||v||| 
with u = <f>x, (b) is due to the RIP, and (c) is from that 

(1 + t)(1 + S) < (1 + r) 2 < 1 + 3 r. 


Hence, we can rewrite (39) as 




< 


< 

< 

< 


K 

IIyIII — (f - 7r) (Xif 

k=\rK ~\-\-1 
K K 

( 1 + 3r)^](a;i) 2 - J2 ( x ~ 7r) {xif 

2—1 i=\rK~\ +1 

+ (1 + 1/ T ) II v||I 

K \tK\ 

IOt ^]^) 2 + O ^) 2 + C 1 + V T ) IMIl 

2=1 2=1 


(10 tK + [rA"])(x max ) 2 + (1 + 1/r) 11v11 2 
11 I’tA’] (x max ) 2 + (1 + 1/r) 11v11 2 - (41) 


Using (41) and (42), we have 




< 


(a) 

< 


< 


< 


1-2 T 

1 

1 — 2r 
1 


11\tK] (x max ) 2 + — 11v11 2 
(11 \tK] (x max ) 2 + 2 t 2 (1 +T 2 )A'(x min ) 2 ) 
(11 [tAT1k 2 + 2t 2 (1 + t 2 )K) (x min ) 2 
\tK\ (x min ) 2 

(43) 


1 - 2r 
11 k 2 + 2r(l + r 2 ) 


1 — 2r 
< Cn 2 TK(x m in) 2 
= C^S^K^) 2 , 

where (a) is due to the facts 1) that 


and 2) that 


SNR - II<&X|12 > k 2 S~ 3/2 - — 

ll,rl|2 - K °2 K _3 1 


|$x|| 2 f (H-5)||x||l 

< (1 + <5)A'(x max ) 2 

< (1 + 5)K(KX min ) 2 , 


and hence 


-||v|| 2 < 2r 2 (l + S)K(x ni i n ) 2 
= 2r 2 (l + T 2 )A(x min ) 2 . 

Finally, by noting that 

||x r ftr || 2 > |r X |(x min ) 2 , 

we have that the number of missed detections satisfies 

|r K | < Ck 2 5 1/2 K. (44) 


On the other hand. 

Ill'll 2 = ||<&(x-x A )+v|| 2 

> (1 - u)||$( x - x K )\\l - (1/r - 1) ||v||l 

(b) 

> (l-r)(l-5)||x-x^|| 2 -(l/r-l)||v|| 2 

> (1 - 2r)||x - x^Hl - (1/r - 1) ||v||l 

> (l-2r)||(x-x^) rK || 2 -(l/r-l)||v|| 2 

( d ) 

> (l-2r)||x r x|| 2 -(l/r-l)||v|| 2 , (42) 

where (a) uses the fact that 

IIu + v|| 1 > (1 — t)||u|| 2 — (1/r — 1) ||v||l 

with u = <I>(x —x A ), (b) follows from the RIP, (c) is because 

(1 — r)(l — S) > (1 — t) 2 = 1 — 2t + r 2 > 1 — 2r, 

and (d) is due to the fact that x A is supported on T K and 
hence x^ K = x A ^ rK = 0. 


Recall that the number missed detections (after K iterations) 
is essentially equal to the number of false alarms (i.e., Y K \ = 
\T K \T\). Therefore, the error rate of support recovery with 
OMP satisfies 


Perror — 


|T A \T| 

m 


jrf[ 

AT 


< Ck 2 ^' 2 . 


(45) 


The proof is now complete. 


V. Conclusion 

In this paper, we have studied the performance of OMP 
for the support recovery of sparse signals under noise. In the 
first part of our analysis, we have shown that in order for the 
OMP algorithm to accurately recover the support of any K- 
sparse signal, the SNR must be at least proportional to the 
sparsity level AT of the signal. For high-dimensional setting, 
our result indicates that the exact support recovery with OMP 
is not possible under finite SNR. 

In the second part of our analysis, we have considered a 
practical scenario where the SNR is an absolute constant in¬ 
dependent of the sparsity K. While the exact support recovery 
with OMP is not possible for this scenario, our analysis has 
shown that recovery with an arbitrarily small but constant 
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fraction of errors is possible. For high-dimensional setting, 
our result offers an affirmative answer to the open question of 
whether OMP can perform the approximate support recovery 
of sparse signals with bounded SNR [12]. We would like to 
point out a technical limitation in this result. Unlike existing 
results for the exact support recovery that depend on the min¬ 
imum magnitude of nonzero elements in the signal, our result 
for the approximate support recovery exhibits the dependence 
on the minimum as well as the maximum magnitudes (more 
precisely, the ratio k). Deriving a similar result but without the 
dependence on the maximum magnitude would require a more 
refined analysis and our future work will be directed towards 
this avenue. 


Appendix A 
Proof of Theorem 3.2 


Appendix B 

Proof of Proposition 3.3 

Proof: We first give a proof of (24). Since u is the largest 
value in {|(&, r k )\} i&T \ T k, 

u = max \{(f>i, r fc )| 
ieT\T k 


> 


(a) 


(b) 

> 


1 


i 


■ (^’ rfe > 2 
V iGT\r fc 

*V fcrfe H 2 


yJK-k 

1 

-j== ll* nr ^ T * 


#rx.rtPr»(*X + v) 


1 


'Jff - f, \JI^T\T fc -f > T !t ^ ?x ll 2 II^T\T fc ^ > T , ' v ll 2 




(47) 


Proof: To prove the necessity of the lower bound of SNR 
in (12), it suffices to show that OMP may fail to recover the 
support of some sparse signal x when 


VSNR< 


VK(1 + 5k+ i) 

(i - Vks k+ i) • Vmar' 


(46) 


In the following, we will show that there exists a set of <&, 
x, and v, for which (46) is satisfied but OMP fails to recover 
the support of x. 

Consider an identity matrix a K'-sparse signal x € 

lZ m with all nonzero elements equal to one, and an 1-sparse 
noise vector v £ 7 Z m as follows. 


where (a) is from (22) and (b) is due to the triangle inequality. 
Observe that 


ll^r\ r fcP r^ X |l 2 

— ll^r\T fcP r fc ^T\T fcX T\T fc ll2 

(b) ||xI 7 -^ 7 - fc 4 , ^ 7 - s .P^ fc #7-y7-fcX7-\7-fe ||2 

H X T\T fc II 2 

(c) II x 7"\7"fc *^T\T k (^ > T fc ) P T'' $ nT fcX T\T fc l|2 

ll x r\r fc II 2 

ll p r fc ^r\r fcX r\r fc ll 2 

ll x r\r fc II 2 





' 1 ' 



' 1 



- 

$ = 

1 

, x = 

1 

0 

, and v = 


1 


0 

- 


> A min ((Prfc^T\T fc ) , Pr fc ^'AT fc ) 11 x r\r fc 111 

11 x r \r fc 11 2 

— ^min ( &T\T kP T k ®T\T k ) ll x r\r fc II 2 

(e) / 

> ^min{^rur k ^ruT k )W x r\r k h 

(/) 

> (1 — <5^) 11 x r \ r fc 11 2 , (48) 


Then the measurements are given by 

' 1 " 


y = 


1 

0 


where (a) is because P.j- t 4*= 0, (b) is from the norm 
inequality, (c) and (d) use the fact that P= (P^) 7 = 
(P^ fc ) 2 , (e) is from Lemma 2.5, and (f) is from the RIP. (Note 
that T k C T and so |T U T k \ = \T\ = K.) Also, 

^r\r fc Pr fcV „ 


0 

_ 1 _ 

In this case, we have Sk+ i = 0, 

||*x || 2 

SNR = „'' 2 = K and MAR = 1. 

IMI 2 

It is easily verified that condition (46) is satisfied; however, 
OMP fails to recover the support of x. Specifically, OMP is 
not guaranteed to make a correct selection at the first iteration. 


- (P^ fc $ rvrfc ) v 2 

< \j Amax ((P^fc^r\r fc ) Pr‘ $ T\r fc ) Il v ll2 

= p^i^)iMi 2 

(c) j ---— 

< Y Amax (^ > ^ uT fc^ rUr fc ) II v|| 2 

< Vl + &K IMI 2 (49) 

where (a) and (b) are due to P^ fc = (P ^ k )' = (P^ fe ) 2 and 
(c) is from Lemma 2.5. 
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Using (47), (48) and (49), we have 

U > _ k ((1 - 6 K )\\xr\r k h - V 1 + ^k ||v|| 2 ) ■ 

(50) 

Next, we proceed to prove (24). Let w £ fl de¬ 
note the index corresponding to the largest element in 

{K0i> rfc )l} ie n\ (r ur*)- Then b y the definition of v > 

v = |(<^,r fc )| 

= |(0 I „,P^(^ X + v ))l 

< K^p^^x)! + |<0 W , p^ fe v)| 

— Kfun f*T k ^T\T kX T\T k )\ + \{^w,Pr kV )\^ (^1) 
where (a) is from the triangle inequality and (b) is because 

P ^k^r k — o. 

Since w T and also noting that T k C T (by the induction 
hypothesis), we have 

|{tn}uT fc U(T\T fc )| = \{w}UT\ = K + l. 

Using this together with Lemma 2.3, we have 

\(4 > w,'Pr k ^T\T kX T\r k )\ 

= ||^ ), u;(Pr !t ^ , r\r fc ) x r\r ''|| 2 

< ^|{w}ur fc u(r\r fe )|ll x r\r fc ll 2 
= ^tf+l|| x T\T' 1 II2 (52) 

and 


where (a) is from that 

x fc = argmin ||y — <f>u|| 2 

u:supp(u)—l~ k 

and hence <I?x fc = ^■ 7 -k^>l / - k y = P-j-iy, (b) is because 
span(Q-j-k) C span{<&q-k+i) so that P^y = Pp k +i (P-T-ity). 

Since span{^^k+ 1 ) D span((j> t k+i), we have 
IIr fc — r fc_l_1 1|| = || P T fc+i r fe ||§ > || P t fe+i r fc ||§. 


Further, by noting that || 

r k _ r fc + 1 ||2 _ ||r fc || 2 — ||r fc+1 || 2 , 

we have 


l|r fc ||l - > 

l|Pt^r fe ||l 

(a) 

ii(^ fc+l) >; fc+1 r fe iil 

> 

m 

(0; fe+1 r fc ) 2 ||0+ fe+1 ||l 

0) 

II^HLll^+alli 

(d) 

> 



(e) 

r5n 

1 , jc > (56 > 

> 


where (a) is because P t k+i = P' fe+1 = (4>\ k+i y<j)' tk+1 , (b) is 
from that <f>' tk+1 r k is a scalar, (c) is due to the identification 
rule of OMP, (d) is from the norm inequality, and (e) follows 
from the RIP. 


P^v)| = |((P^)V™,v}| 

= |<P^,v>| 

(b) 

< UPr^^lb ||v|| 2 

< || 4*w || 2 11V11 2 

(c) 


where (a) is due to the fact that P^- k = (P^ fc )', (b) is from 


< Vl + Si ||v|| 2) (53) 

^k = (P^)', 

Cauchy-Schwarz inequality, and (c) is from the RIP. 

Finally, combining (51), (52), and (53) we have 

v < fe+i||x T \ r fe|| 2 + \/l + <5i ||v|| 2 , (54) 

which completes the proof. ■ 

Appendix C 

Proof of (35) 

Proof: First, from the definition of OMP (see Table I), 
we have that for any integer k > 0, 

r fc_ r fc+l = (y_$x fe )-(y-$x fc+1 ) 

= Pr*y-Pr^y 

— (P 7 - 1+1 — Pyi+iPy^y 

= P T !+iP^iy 

= P T i+ir k , 


Appendix D 
Proof of (36) 

Proof: The main goal of the proof is to establish a lower 
bound on ||$'r fc ||^ 0 . 

Observing that 

(&r k ) T k = & Tk (P^ k y) = (P^ rk )'y = 0, 
we have 


$'r fc ||^ = ||(^r fe ) nvrfc || 


(57) 


Then it follows from the Holder’s inequality that for all w £ 

7 

P'r%\r‘.Wfi\ri -) 2 


$'r fc || 2 > 


II w o\r fc 111 

(3>'r fe , w) 2 

II w o\r fc Hi 

(: r k , *I>w) 2 


ll w o\T fc lli 

Let w £ 7 Z n be a vector such that 


(58) 


( 55 ) 


Ws = h 5CTnT‘ur5, 

\o S cn\(T(lT k UT k ). 

where r = (5 1 / 2 . See Figure 1 for an illustration of supp(w). 
Then (58) can be rewritten as 
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\\*'r k \L > 

> 

(а) 
> 

( б ) 


(4> , r fc , w) 2 
llxrfll? 

(r fc , <Fw) 2 
ll x r£ Ilf 
(r fc , *I>w) 2 

MnI 

(r fc , $w) 2 

r^nix r ,n 2 

(r fe , <F(w — x fe )} 2 

' ^All|x r ,||i 


(60) 


Further, observe that 

ll$(x-x fc )|| 2 - ||$(w-x)|| 2 

- II^(x x^)|| 2 - ||$ rfc \rjx rfe \rj||i 
( 6 ) 

> (1 — ^)ll x — x fc III — (1 + <5)||x r fcy r fc ||| 

> (l-«5)||(x-x fc ) r ,||i-(l + <5)||x rfe \ r j|| 2 

— (1 — ^)ll x r fc ll2 — (1 + <5)ll x r fc \rfc II1 
= (1 ~ (ll x rfc 111 + ll x r fc \rj III) 

(1 + ^)ll x r fe \r£ III 

= (1 — ^)||x r fc III — 2(5||x r fcy r fc III 

( d ) 

> (1 — 5)\tK~\ (x k ) 2 — 2SK(x k ) 2 

> (l-6-^\rK](x k T ) 2 

> (1 — 3r) [risT] (x k ) 2 , (62) 

where (a) is from (59), (b) is due to the RIP, (c) is because 
Xp fc = 0, (d) is because x k is the \tK~\- th largest element (in 
magnitude) in x r t : and hence is the smallest one in Xp*, and 
(e) is from that 


where (a) follows from the norm inequality (||u||i < 

y/\\ u||o 11 u 11 2 ) and (b) is because 

(r fe ,$x fe ) = (P^y,P rfc y) = (y,P^ fc P r *y) = 0. 

We now consider the term (r , ‘l>('w — x fc )) 2 in the numer¬ 
ator of the right-hand-side of (60). Since the residual can be 
expressed as r k = <I>(x — x fc ) + v, we have 

(r fc ,$(w-x fc )} 2 

= (#(x-x fc )+v,$(w-x fe )) 2 

= «*(x - x fe ), #(w - x fc )) + (v, «&(w - x fe )}) 2 

> (1 — r)($(x — x k ), <F(w — x fc )) 2 — (1/r — 1) 

x (v, $(w - x fe )) 2 

( b ) 

> (1 - r)(#(x - x fc ), $(w - x fc )) 2 - (1/r - 1) 
x||v||| ||$(w-x fc )|| 2 

= i T ^(||^( x - xfc )ll|-|l^(w-x)|| 2 

+ ll*(w - xfe )||^) 2 - (1/r - 1) ||v||l ||$(w - x fc )|| 2 , 

(61) 

where (a) uses the inequality 

(it + v) 2 > (1 — t)u 2 — (1/r — 1) v 2 

with it = (<F(x — x fc ), <i>(w — x fc )} and v = (v, $(w — x fe )}, 

(b) is due to the Cauchy-Schwarz inequality, and (c) is from 
the fact that 

( u > z ) = \ (ll u lll - ll u - z lli + IMli) 

with u = d?(x — x fc ) and z = d?(w — x k ). 


1 - <5 -= 1 - 6 - 2t > 1 - 3t. 

T 

Using (61) and (62), we have 
(r fc ,$(w-x fc )) 2 

> ((1 - 3r)|"r A'] {x k T f + ||*(w - x fc )|| 2 ) 2 
-(l/r^l)||v|| 2 ||$(w-x fc )|| 2 

> (1 -r) ((1 -3r)priT|(:r*) 2 ) ||$(w-x fc )|| 2 
-(l/r-l)||v|| 2 ||<I>(w-x fc )|| 2 

( 6 ) 

> ((1 — 4r) \tK~\ (x k ) 2 — (1/r — 1) ||v|| 2 ) 
x ||$(w — x fc ) ||| 

> (l-5r)[ris:l(^) 2 ||^(w-x fc )||^ 

( d ) 

> (1 ~ 6r) \tK~\ (a:*) 2 (l — <5)||w — x fc ||§ 

> (1 — 6r) \tK~\ (x k ) 2 1|w — x fc || 2 

> (l-6r)rr/xl^) 2 ||(w-x fc ) nvr ,|| 2 

= (l-6r)rr/xl^) 2 ||x r ,|| 2 (63) 

where (a) follows from the fact that (it + v) 2 > Auv with 
it = (1 — 3t) \tK~\ (x k ) 2 and v = ||<I>(w — x fc )||(b) is 
because 


(1 — t)( 1 — 3r) = 1 — 4r + 3r 2 > 1 — 4 t, 
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(c) is because 1) the assumption SNR > k 2 <5 -3 / 2 implies that 


l^xll, = SNR • llvll 


> 


— I 

53/2 l 

.,2 




1 — r I t — 



k 2 (1 + r 2 )(l — r) 


and 2) on the other hand. 


„ RIP 

l*x|| 2 < 

< 

< 

< 


(1 + «)W 


(64) 


(1 + 6)Kx 2 max 

(1 + 6)I< (KXmin) 2 

(1 + S)K(kx k T f 

= (1 + t 2 )K(kXt) 2 , (65) 

and hence 

(!/t - 1) ||v||| < t 2 K(x*) 2 < t\tK ] (x^) 2 , (66) 

(d) is from the RIP, and (e) follows from that 
(1 — 5t)(1 — <5) > (1 — 5r)(l — r) = 1 — 6r + 5r 2 > 1 — 6r. 
Finally, using (60) and (63), we have 

(r fe , <&(w — x fe )) 2 


l$ , r fc ||L 


> 


> 


\tK] ||x r *|| 2 

(l-6r)rr^l(^) 2 ||x r ,|| 2 


W||x r ,||| 

= (l-6r)(^) 2 , 

which completes the proof. 


(67) 
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